Add correlations on the scatter plots:

# Customize upper panel
upper.panel<-function(x, y){
  points(x,y, pch=19, col=c("red", "green3", "blue")[iris$Species])
  r <- round(cor(x, y), digits=2)
  txt <- paste0("R = ", r)
  usr <- par("usr"); on.exit(par(usr))
  par(usr = c(0, 1, 0, 1))
  text(0.5, 0.9, txt)
}
pairs(iris[,1:4], lower.panel = NULL, 
      upper.panel = upper.panel)

Use the R package psych

The function pairs.panels [in psych package] can be also used to create a scatter plot of matrices, with bivariate scatter plots below the diagonal, histograms on the diagonal, and the Pearson correlation above the diagonal.

library(psych)

## 
## Attaching package: 'psych'

## The following objects are masked from 'package:ggplot2':
## 
##     %+%, alpha

pairs.panels(iris[,-5], 
             method = "pearson", # correlation method
             hist.col = "#00AFBB",
             density = TRUE,  # show density plots
             ellipses = TRUE # show correlation ellipses
             )

When creating histograms or barplots in ggplot2 we found that the data is placed at some distance from the x axis, which means the y axis starts below zero:

library(tree)

## Registered S3 method overwritten by 'tree':
##   method     from
##   print.tree cli

tree1 <- tree(Species
~Sepal.Width +
Petal.Width, data = iris)
plot(tree1)
text(tree1)

3. Cleaning data through pipes

glimpse(iris)

## Observations: 150
## Variables: 8
## $ Sepal.Length <dbl> 5.1, 4.9, 4.7, 4.6, 5.0, 5.4, 4.6, 5.0, 4.4, 4.9, 5…
## $ Sepal.Width  <dbl> 3.5, 3.0, 3.2, 3.1, 3.6, 3.9, 3.4, 3.4, 2.9, 3.1, 3…
## $ Petal.Length <dbl> 1.4, 1.4, 1.3, 1.5, 1.4, 1.7, 1.4, 1.5, 1.4, 1.5, 1…
## $ Petal.Width  <dbl> 0.2, 0.2, 0.2, 0.2, 0.2, 0.4, 0.3, 0.2, 0.2, 0.1, 0…
## $ Species      <fct> setosa, setosa, setosa, setosa, setosa, setosa, set…
## $ setosa       <lgl> TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, TRUE, TRU…
## $ virginica    <lgl> FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FA…
## $ versicolor   <lgl> FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FALSE, FA…

The output shows 150 variables spread across 6 variables. 4 variable are numeric ones, and species are categorical. Lets move on.

head(iris, n = 10)

##    Sepal.Length Sepal.Width Petal.Length Petal.Width Species setosa
## 1           5.1         3.5          1.4         0.2  setosa   TRUE
## 2           4.9         3.0          1.4         0.2  setosa   TRUE
## 3           4.7         3.2          1.3         0.2  setosa   TRUE
## 4           4.6         3.1          1.5         0.2  setosa   TRUE
## 5           5.0         3.6          1.4         0.2  setosa   TRUE
## 6           5.4         3.9          1.7         0.4  setosa   TRUE
## 7           4.6         3.4          1.4         0.3  setosa   TRUE
## 8           5.0         3.4          1.5         0.2  setosa   TRUE
## 9           4.4         2.9          1.4         0.2  setosa   TRUE
## 10          4.9         3.1          1.5         0.1  setosa   TRUE
##    virginica versicolor
## 1      FALSE      FALSE
## 2      FALSE      FALSE
## 3      FALSE      FALSE
## 4      FALSE      FALSE
## 5      FALSE      FALSE
## 6      FALSE      FALSE
## 7      FALSE      FALSE
## 8      FALSE      FALSE
## 9      FALSE      FALSE
## 10     FALSE      FALSE

What we notice is that the first 4 variables capture Lemght and Witdth measurements for sepal and petal. We need to change to a more easy format.

long_iris <- iris%>%
  gather(part,value,Sepal.Length,Sepal.Width,Petal.Length ,Petal.Width)%>%
  separate(part, c('part', 'measure'), sep = '\\.')

Now we can plot more precise graphics by showing the species, the data stills the same, we just removed some limitations of it original format.

Data Cleaning

sapply(long_iris, class)

##     Species      setosa   virginica  versicolor        part     measure 
##    "factor"   "logical"   "logical"   "logical" "character" "character" 
##       value 
##   "numeric"

Now we coerce character variables to R factors. This will be handy & facilitate ease of analysis.

fcts <- c('part', 'measure')
long_iris[fcts] <- lapply(long_iris[fcts], as.factor)
sapply(long_iris, class)

##    Species     setosa  virginica versicolor       part    measure 
##   "factor"  "logical"  "logical"  "logical"   "factor"   "factor" 
##      value 
##  "numeric"

Lets write a script that will loop over the entire data set to get a sum of each missing value of each variable

Missing_d <- function(x){sum(is.na(x))/length(x)*100}

Lets apply to long_iris our missing_d

apply(long_iris, 2, Missing_d)

##    Species     setosa  virginica versicolor       part    measure 
##          0          0          0          0          0          0 
##      value 
##          0

Now we can see that are zero missing values

is_special <- function(x){
  if(is.numeric(x)) !is.finite(x) else is.na(x)
}

By using the sapply() function it performs a check on variable numerics to see if theres some NA.

sum(is.na(long_iris$value))

## [1] 0

3. New long_iris dataset cleaned. We used some “pipes” to perform a change on the way it was formated, used a script to get rid of NA values.

p <- ggplot(long_iris, aes(x = Species, y = value, col = part)) 
p + geom_jitter(alpha = 0.4, size = 0.8) + facet_grid(.~ measure)

With our data cleaning, now we are able to show our species.

p + geom_jitter(alpha = 0.3, size = 0.8) + stat_boxplot(alpha = 0.5) + facet_grid(.~ measure)

Remember our first box plot? now we dont need to guess from where some variable came from.

p + geom_jitter(alpha = 0.5, size = 0.8) + stat_boxplot(alpha = 0.5) + facet_grid(.~ part)

Observations on each species/ Conclusion

Setosa’s petal & sepal length are the shortest of the 3 species, their petals are the narrowest compared to the other 2 species but notice how their sepal is the widest of the 3. Versicolor and Virginica are similar in length and width properties. Setosa has less petals but notice something interesting on his sepals, which shows more sepals that the versicolor and virginica, even if it appears that the other 2 species have more. Versicolor and virginica have more spread between max and min.

lets create a new data object with length and width as variables:

iris$Flower <- 1:nrow(iris)

#create wide_iris
wide_iris <- iris %>%
  gather(key, value, -Species, -Flower) %>%
  separate(key, c("Part", "Measure"),sep = "\\.") %>%
  spread(Measure, value)

## Warning: Expected 2 pieces. Missing pieces filled with `NA` in 450 rows
## [601, 602, 603, 604, 605, 606, 607, 608, 609, 610, 611, 612, 613, 614, 615,
## 616, 617, 618, 619, 620, ...].

head(wide_iris, n = 10)

##    Species Flower       Part Length Width <NA>
## 1   setosa      1      Petal    1.4   0.2   NA
## 2   setosa      1      Sepal    5.1   3.5   NA
## 3   setosa      1     setosa     NA    NA    1
## 4   setosa      1 versicolor     NA    NA    0
## 5   setosa      1  virginica     NA    NA    0
## 6   setosa      2      Petal    1.4   0.2   NA
## 7   setosa      2      Sepal    4.9   3.0   NA
## 8   setosa      2     setosa     NA    NA    1
## 9   setosa      2 versicolor     NA    NA    0
## 10  setosa      2  virginica     NA    NA    0

Now lets investigate the relashionship between lenght and width by species. Lets fit a non-predictive and a predictive linear model to analyze the relashionship and see if the linear model do a good job at fitting the data and describe the lenght.

q <- ggplot(wide_iris, aes(x = Width, y = Length, col = Species))
q + geom_jitter(alpha = 0.4, size = 0.8) + facet_grid(. ~ Species) + 
  stat_smooth(method = 'lm', se = F)

## Warning: Removed 450 rows containing non-finite values (stat_smooth).

## Warning: Removed 450 rows containing missing values (geom_point).

q + geom_jitter(alpha = 0.4, size = 0.8) + facet_grid(. ~ Part)

## Warning: Removed 450 rows containing missing values (geom_point).

q + geom_point(alpha = 0.4, size = 0.8)  + stat_smooth(method = 'lm', fullrange = T, size = 0.5)

## Warning: Removed 450 rows containing non-finite values (stat_smooth).

## Warning: Removed 450 rows containing missing values (geom_point).

Conclusion

Student dots: In this project i was able to apply some skills that ive acquired through the class, and learned something outside the class like shiny apps(yes, shinny apps makes R ready for the future). Ive tryed to create a good project based on iris dataset (since we was able to use any dataset on the links provided), which showed very interesting for me, since i was able to work with boxplot and understand better how it works (since its more oriented to biostatistics, i guess). I can see data data science are awesome, but is not a easy task. I see that the content are great, but theres a lot to learn in order to be a professional (yes, we are very far). Theres a lot of concepts that now i understand much better than the beggining, but theres a long path that i will follow in order to master my skills on the subject. Im very happy with the results, since for a begginner class, ive learned a lot.

About the project:

On this project ive used every tool that i was able to put on practice, in order to create a good report on data science related to the Iris dataset. I tryed to explore plots(graphics) at the maximum that i could. At the beginning i used the “raw” data and explored with plots, them ive used our sugar (%>%) from tidyverse library to format the library and plot some more precise graphics on each species making possible to visualize better; since at start all was about lenght and widht.

My conclusion, after this plots are:

Setosa’s petal & sepal length are the shortest of the 3 species, their petals are the narrowest compared to the other 2 species but notice how their sepal is the widest of the 3. Versicolor and Virginica are similar in length and width properties. Setosa has less petals but notice something interesting on his sepals, which shows more sepals that the versicolor and virginica, even if it appears that the other 2 species have more. Versicolor and virginica have more spread between max and min.

thanks to Mr. Rafael Irizzary to show the R path, which now i think that is better than python, i hope to have more classes or a part 2 of the harvard data science course on edx.